All kinds of computers such as a PC, a workstation, a server, a rooter, a cellular telephone and a PDA are always exposed to attacks from the outside and the inside. A typical attack is one that takes advantage of vulnerability of software being executed on a computer. An attacker sends a malicious code to a computer, which exploits vulnerability of the software, so as to take control of a process being executed. Then, the attacker abusively manipulates the computer by utilizing the authority of the process.
In order to detect the attack that exploits vulnerability, especially, to detect an unknown attack, an anomaly detection system can be considered. In this system, the normal behavior of software is modeled in advance. Then, it is determined whether there is a difference between the normal behavior and the behavior of software being executed. A system call is an instruction made by a process when the process requests a kernel to perform processing important for the system. An attacker exploits a system call requested by a process, and thereby causes the system to perform an arbitrary behavior intended by the attacker. For this reason, when the behavior of software is monitored, the validity of a system call is examined (e.g., refer to JPA 2004-126854 (hereinafter referred to as JPA 2004-126854); Feng, H. et al., “Anomaly Detection Using Call stack Information,” The proc. of IEEE Symposium on Security and Privacy 2003, p. 62.; and Abe et al., “Seiteki_Kaiseki_ni_Motoduku_Sinnyuu_Kenchi_Shisutemu_no_Saitekika (Optimization of Intrusion Detection System Based On Static Analysis),” IPSJ (Information Processing Society of Japan) Journal, Vol. 45, No. SIG3(ACS 5), pp. 11-202).
An attack countermeasure device described in JPA 2004-126854 includes a system call table, a validity determination function unit, an attack countermeasure function unit and a system call. In the device, it is judged as normal when a function of a system call issuer locates in a code region of a memory. The system call table receives a system call request issued from the software (a task), and then outputs a jump address for calling a validity determination function. The validity determination function unit determines the validity of the system call request based on a return address of a system call issuer, the return address being stored in a predetermined region of the memory by the OS when the system call request is issued. The validity determination function unit then outputs the result of the determination. When the validity determination function unit determines that the system call request is invalid, the validity determination function unit discards the system call request. The attack countermeasure function unit receives the result of the determination on the invalid system call request made by the validity determination function unit, and then takes a countermeasure. The system call is called upon receipt of the result of the determination on the invalid system call request made by the validity determination function unit, and causes an instruction to be executed.
In an attack detection system described in “Anomaly Detection Using Call stack Information,” the state of a call stack (a column of return addresses stacked in stack) is utilized in order to verify the validity of a system call. In this system, first, software (a program) is executed in advance, and the system learns a model from the result obtained by the execution. The system obtains the state of a call stack at the time when a system call is made during the execution of the program, and generates a virtual stack list in which the state of the call stack and the program counter at the time of the system call issuance are recorded. In addition, the system generates information on a difference between the present virtual stack list and the previous virtual stack list. That is, the system sequentially compares and verifies the state of the call stack in the present virtual stack list, which is a target for comparison, with the state of the call stack in the previous virtual stack list from the lowermost stack level, and thereby detects a different return address therebetween. Thereafter, the system generates a column of return addresses subsequent to the detected different return address (virtual path). The system generates hash tables respectively from the generated virtual stack list and virtual path so as to utilize the generated hash tables as the model for the software. When the system verifies the behavior of the software, the system firstly generates the virtual stack list and the virtual path during the execution of the software. Then, the system performs matching for the generated virtual stack list and virtual path, with and the respective hash tables, which are of the hash tables as the model for the software. When the generated value matches the value of the hash table, the system gives permission to the system call request. Meanwhile, if the generated value does not match the value of the hash table, the system determines that the value is anomaly.
In an attack detection system described in “Optimization of an intrusion detection system based on static analysis,” the state of a call stack (a column of return addresses stacked in stack) is utilized in order to verify the validity of a system call. This system analyzes a binary code of software to be monitored so as to generate function definitions and automatons for the function definitions. These automatons are generated respectively for functions, and are used for monitoring the control flow of the software. A node is defined as one of (1) entry of function, (2) exit of function, and (3) call of function. During the execution of software to be monitored, this system suspends the software for each time a system call is issued, so as to check the state of the call stack at the time. Then, the system creates, from the column of return addresses, a list of functions (referred to as a stack back trace) that have been called. When i is set as a natural number, the system compares the (i−1)-th stack back trace and the i-th stack back trace, and verifies whether or not a control flow from the former stack back trace to the latter stack back trace locates in the models (the automatons).
In an Operating System such as Linux, a normal system call is issued through a Wrapper function. Since the Wrapper function locates in a code region, a function of a system call issuer always locates in the code region. In addition, in the case of a Return-to-libc attack that is a major attack, an attacker causes the system to return to libc so that an arbitrary system call is issued. Considering the above situation, an attack that cannot be detected by the attack countermeasure device described in JPA 2004-126854 is located, since in the device described in JPA 2004-126854, a system is validated when the address of a function of a system call issuer locates in a code region. Here, the attack detection systems described in both “Anomaly Detection Using Call stack Information” and “Optimization of an intrusion detection system based on static analysis” make verification by using a column of return addresses stacked in call stack. Accordingly, it can be said that these systems perform more detailed modeling than the attack countermeasure device of JPA 2004-126854 does. For this reason, false negatives can be reduced more than in the case of JPA 2004-126854.